What is a string?

A string is a bunch of characters.

There is a difference between…

my_string <- "Hi, my name is Bond!"
my_vector <- c("Hi", "my", "name", "is", "Bond")

my_string

[1] "Hi, my name is Bond!"

my_vector

[1] "Hi"   "my"   "name" "is"   "Bond"

stringr

pattern =

The pattern argument appears in many stringr functions .

• The pattern must be supplied inside quotes.

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")

str_detect(my_vector, pattern = "Bond")
str_locate(my_vector, pattern = "Bond")
str_match(my_vector, pattern = "Bond")
str_extract(my_vector, pattern = "Bond")
str_subset(my_vector, pattern = "Bond")

str_detect()

Returns a logical vector indicating whether the pattern was found in each element of the supplied vector.

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_detect(my_vector, pattern = "Bond")

[1] FALSE FALSE  TRUE  TRUE

str_match()

Returns a character matrix containing either NA or the pattern, depending on if the pattern was found.

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_match(my_vector, pattern = "Bond")

     [,1]  
[1,] NA    
[2,] NA    
[3,] "Bond"
[4,] "Bond"

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_match(my_vector, pattern = "(.)o(.)")

     [,1]  [,2] [,3]
[1,] "lo," "l"  "," 
[2,] NA    NA   NA  
[3,] "Bon" "B"  "n" 
[4,] "Bon" "B"  "n" 

str_extract()

Returns a character vector with either NA or the pattern, depending on if the pattern was found.

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_extract(my_vector, pattern = "Bond")

[1] NA     NA     "Bond" "Bond"

str_locate()

Returns a dateframe with two numeric variables – the starting and ending location of the pattern.

• The values are NA if the pattern is not found.

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_locate(my_vector, pattern = "Bond")

     start end
[1,]    NA  NA
[2,]    NA  NA
[3,]     1   4
[4,]     7  10

str_subset()

Returns a character vector containing a subset of the original character vector consisting of the elements where the pattern was found.

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_subset(my_vector, pattern = "Bond")

[1] "Bond"       "James Bond"

Try it out!

my_vector <- c("I scream,", "you scream", "we all",
               "scream","for","ice cream")

str_detect(my_vector, pattern = "cream")
str_locate(my_vector, pattern = "cream")
str_match(my_vector, pattern = "cream")
str_extract(my_vector, pattern = "cream")
str_subset(my_vector, pattern = "cream")

Replace / Remove Patterns

str_replace(my_vector, pattern = "Bond", replace = "Franco")

[1] "Hello,"       "my name is"   "Franco"       "James Franco"

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_remove(my_vector, pattern = "Bond")

[1] "Hello,"     "my name is" ""           "James "    

Edit Strings

Convert letters in a string to a specific capitalization format.

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_to_lower(my_vector)

[1] "hello,"     "my name is" "bond"       "james bond"

str_to_upper(my_vector)

[1] "HELLO,"     "MY NAME IS" "BOND"       "JAMES BOND"

str_to_title(my_vector)

[1] "Hello,"     "My Name Is" "Bond"       "James Bond"

Combine Strings

prompt <- "Hello, my name is"
first  <- "James"
last   <- "Bond"
str_c(prompt, last, ",", first, last, sep = " ")

[1] "Hello, my name is Bond , James Bond"

my_vector <- c("Hello,", "my name is", "Bond", "James Bond")
str_flatten(my_vector, collapse = " ")

[1] "Hello, my name is Bond James Bond"

first <- "James"
last <- "Bond"
str_glue("My name is {last}, {first} {last}")

My name is Bond, James Bond

Tips for String Success

• Refer to the stringr cheatsheet

• Remember that str_xxx functions need the first argument to be a vector of strings, not a dataset!

  • You will use these functions inside dplyr verbs like filter() or mutate().

cereal |> 
  mutate(is_bran = str_detect(name, "Bran"), 
         .after = name)

Tips for String Success

The real power of these str_xxx functions comes when you specify the pattern using regular expressions!

Regular Expressions

“Regexps are a very terse language that allow you to describe patterns in strings.”

R for Data Science

str_detect(string  = my_string_vector,
           pattern = "p[ei]ck[a-z]")

Regular Expressions

Regular expressions are tricky!

• There are lots of new symbols to keep straight.
• There are a lot of cases to think through.

This web app for testing R regular expressions might be handy!

Special Characters

There is a set of characters that have a specific meaning when using regex.

• The stringr package does not read these as normal characters.
• These characters are:

Wild Card Character: .

. – matches any character.

x <- c("She", "sells", "seashells", "by", "the", "seashore!")
str_subset(x, pattern = ".ells")

[1] "sells"     "seashells"

This matches strings that contain any character followed by “ells”.

Anchor Characters: ^ $

^ – looks at the beginning of a string.

x <- c("She", "sells", "seashells", "by", "the", "seashore!")
str_subset(x, pattern = "^s")

[1] "sells"     "seashells" "seashore!"

This matches strings that start with “s”.

str_subset(x, pattern = "s$")

[1] "sells"     "seashells"

Quantifier Characters: ? + *

? – matches when the preceding character occurs 0 or 1 times in a row.

x <- c("shes", "shels", "shells", "shellls", "shelllls")
str_subset(x, pattern = "shel?s")

[1] "shes"  "shels"

str_subset(x, pattern = "shel+s")

[1] "shels"    "shells"   "shellls"  "shelllls"

str_subset(x, pattern = "shel*s")

[1] "shes"     "shels"    "shells"   "shellls"  "shelllls"

Quantifier Characters: {}

{n} – matches when the preceding character occurs exactly n times in a row.

x <- c("shes", "shels", "shells", "shellls", "shelllls")
str_subset(x, pattern = "shel{2}s")

[1] "shells"

str_subset(x, pattern = "shel{2,}s")

[1] "shells"   "shellls"  "shelllls"

str_subset(x, pattern = "shel{1,3}s")

[1] "shels"   "shells"  "shellls"

Character Groups: ()

Groups are created with ( ).

• We can specify “either” / “or” within a group using |.

x <- c("Peter", "Piper", "picked", "a", "peck", "of", "pickled", "peppers!")
str_subset(x, pattern = "p(e|i)ck")

[1] "picked"  "peck"    "pickled"

This matches strings that contain either “peck” or “pick”.

Character Classes: []

Character classes let you specify multiple possible characters to match on.

x <- c("Peter", "Piper", "picked", "a", "peck", "of", "pickled", "peppers!")
str_subset(x, pattern = "p[ei]ck")

[1] "picked"  "peck"    "pickled"

str_subset(x, pattern = "p[^i]ck")

[1] "peck"

str_subset(x, pattern = "^p")

[1] "picked"   "peck"     "pickled"  "peppers!"

str_subset(x, pattern = "^[Pp]")

[1] "Peter"    "Piper"    "picked"   "peck"     "pickled"  "peppers!"

Character Classes: []

[ - ] – specifies a range of characters.

x <- c("Peter", "Piper", "picked", "a", "peck", "of", "pickled", "peppers!")
str_subset(x, pattern = "p[ei]ck[a-z]")

[1] "picked"  "pickled"

Shortcuts

\\w – matches any “word” (\\W matches not “word”)

• A “word” contains any letters and numbers.

\\d – matches any digit (\\D matches not digit)

\\s – matches any whitespace (\\S matches not whitespace)

• Whitespace includes spaces, tabs, newlines, etc.

x <- "phone number: 1234567899"
str_extract(x, pattern = "\\d+")

[1] "1234567899"

str_extract_all(x, pattern = "\\S+")

[[1]]
[1] "phone"      "number:"    "1234567899"

Try it out!

What regular expressions would match words that…

• end with a vowel?
• start with x, y, or z?
• do not contain x, y, or z?
• contain British spelling?

x <- c("zebra", "xray", "apple", "yellow",
       "color", "colour", "summarize", "summarise")

str_subset(x, "[aeiouy]$")
str_subset(x, "^[xyz]")
str_subset(x, "^[^xyz]+$")
str_subset(x, "(our)|(i[sz]e)")

Escape: \\

To match a special character, you need to escape it.

x <- c("How", "much", "wood", "could", "a", "woodchuck", "chuck",
       "if", "a", "woodchuck", "could", "chuck","wood?")
str_subset(x, pattern = "?")

Error in stri_subset_regex(string, pattern, omit_na = TRUE, negate = negate, : Syntax error in regex pattern. (U_REGEX_RULE_SYNTAX, context=`?`)

str_subset(x, pattern = "\\?")

[1] "wood?"

str_subset(x, pattern = "[?]")

[1] "wood?"

When in Doubt

Use the web app to test R regular expressions.

Tips for working with regex

• Read the regular expressions out loud like a request.

str_view(c("shes", "shels", "shells", "shellls", "shelllls"), "l+")

[2] │ she<l>s
[3] │ she<ll>s
[4] │ she<lll>s
[5] │ she<llll>s

stringr functions + dplyr verbs!

Country names with a (capital or lowercase) “Z”?

military |> 
  filter(str_detect(Country, "[Zz]")) |> 
  distinct(Country)

military |> 
  distinct(Country) |> 
  summarize(prop = mean(str_detect(Country,
                                   "[Nn]orth|[Ss]outh|[Ee]ast|[Ww]est")))

# A tibble: 1 × 1
    prop
   <dbl>
1 0.0789

matches(pattern)

Slect all variables with a name that matches the supplied pattern.

• Pairs well with select(), rename_with(), and across().

military_clean <- military |> 
  mutate(across(`1988`:`2019`, 
                ~ na_if(.x, y = ". .")),
         across(`1988`:`2019`, 
                ~ na_if(.x, y = "xxx")))

military_clean <- military |> 
  mutate(across(matches("[1-9]{4}"), 
                ~ na_if(.x, y = ". .")),
         across(matches("[1-9]{4}"), 
                ~ na_if(.x, y = "xxx")))

Messy Covid Variants!

What is that variable?!

PA 5.1: Scrambled Message

In this activity, you will use regex to decode a message.

• Remember: stringr functions go inside dplyr verbs like mutate() and filter() – use them like as.factor.

x <- c("She", "sells", "seashells", "by", "the", "seashore!")

x[c(1,4,5)]

[1] "She" "by"  "the"

x[c(1,4,5)] <- ""

To do…

• PA 5.1: Scrambled Message
  • Due Saturday, 5/4 at 11:59pm
• Final Project Group Formation Survey
  • Due Friday, 5/3 at 11:59pm

Wednesday, May 3

Today we will…

• Midterm Exam 5/8: What to Expect
• New Material
  • Working with Date & Time Variables
• PA 5.2: Jewel Heist
• LA 5: Murder in SQL City

Midterm Exam – Wednesday, 5/8

• This is a three-part exam:
  1. You will first complete a General Questions section on paper and without your computer.
  2. After you turn that in, you will complete a Short Answer section with your computer.
  • You will have the one hour and 50 minute class period to complete the first two sections.
  3. The third section, Open-Ended Analysis, will be started in class and due 24 hours after the end of class.

Midterm Exam – Wednesday, 5/8

• The exam is worth a total of 100 points.
  • Approx. 20 pts, 30 pts, and 50 pts for the three sections.
• I will provide a .qmd template for the Short Answer.
• You will create your own .qmd for the Open-Ended Analysis. You are encouraged to create this ahead of time.

Why are dates and times tricky?

When parsing dates and times, we have to consider complicating factors like…

• Daylight Savings Time.
  • One day a year is 23 hours; one day a year is 25 hours.
  • Some places use it, some don’t.
• Leap years – most years have 365 days, some have 366.
• Time zones.

lubridate

library(lubridate)

date-time Objects

There are multiple data types for dates and times.

• A date:
  • date or Date
• A date and a time (identifies a unique instant in time):
  • dtm
  • POSIXlt – stores date-times as the number of seconds since January 1, 1970 (“Unix Epoch”)
  • POSIXct – stores date-times as a list with elements for second, minute, hour, day, month, year, etc.

Creating date-time Objects

Create a date from individual components:

make_date(year = 1995, month = 05, day = 08)

[1] "1995-05-08"

mdy("May 8, 1995")

[1] "1995-05-08"

dmy("8-May-1995", tz = "America/Chicago")

[1] "1995-05-08 CDT"

dmy_hms("8-May-1995 9:32:12", tz = "America/Chicago")

[1] "1995-05-08 09:32:12 CDT"

as_datetime("95-05-08", format = "%y-%m-%d")

[1] "1995-05-08 UTC"

parse_datetime("5/8/1995", format = "%m/%d/%Y")

[1] "1995-05-08 UTC"

Creating date-time Objects

Common Mistake with Dates

What’s wrong here?

as_datetime(2023-02-6)

[1] "1970-01-01 00:33:35 UTC"

my_date <- 2023-02-6
my_date

[1] 2015

Extracting date-time Components

bday <- ymd_hms("1993-11-20 9:32:12", tz = "America/New_York")
bday

[1] "1993-11-20 09:32:12 EST"

year(bday)

[1] 1993

month(bday)

[1] 11

day(bday)

[1] 20

wday(bday)

[1] 7

wday(bday, label = TRUE, abbr = FALSE)

[1] Saturday
7 Levels: Sunday < Monday < Tuesday < Wednesday < Thursday < ... < Saturday

Subtraction with date-time Objects

Doing subtraction gives you a difftime object.

• difftime objects do not always have the same units – it depends on the scale of the objects you are working with.

How old am I?

today() - mdy(11201993)

Time difference of 11111 days

How long did it take me to finish a typing challenge?

begin <- mdy_hms("3/1/2023 13:04:34")
finish <- mdy_hms("3/1/2023 13:06:11")
finish - begin

Time difference of 1.616667 mins

Durations and Periods

Durations will always give the time span in an exact number of seconds.

as.duration(today() - mdy(11201993))

[1] "959990400s (~30.42 years)"

as.duration(finish - begin)

[1] "97s (~1.62 minutes)"

as.period(today() - mdy(11201993))

[1] "11111d 0H 0M 0S"

as.period(finish - begin)

[1] "1M 37S"

Durations and Periods

We can also add time:

• days(), years(), etc. will add a period of time.
• ddays(), dyears(), etc. will add a duration of time.

mdy(11201993) + years(99)

[1] "2092-11-20"

mdy(11201993) + dyears(99)

[1] "2092-11-19 18:00:00 UTC"

Time Zones

Time zones are complicated!

Specify time zones in the form:

• {continent}/{city} – “America/New_York”, “Africa/Nairobi”
• {ocean}/{city} – “Pacific/Auckland”

Sys.timezone()

[1] "America/Los_Angeles"

Time Zones

You can change the time zone of a date in two ways:

x <- ymd_hms("2024-06-01 18:00:00", tz = "Europe/Copenhagen")

x |> 
  with_tz()

[1] "2024-06-01 09:00:00 PDT"

x |> 
  with_tz(tzone = "Asia/Kolkata")

[1] "2024-06-01 21:30:00 IST"

x |> 
  force_tz()

[1] "2024-06-01 18:00:00 PDT"

x |> 
  force_tz(tzone = "Asia/Kolkata")

[1] "2024-06-01 18:00:00 IST"

Common Mistake with Dates

When you read data in or create a new date-time object, the default time zone (if not specified) is UTC.

• UTC (Universal Time Coordinated) is the same as GMT (Greenwich Mean Time).

Make sure you specify your desired time zone!

x <- mdy("11/20/1993")
tz(x)

[1] "UTC"

x <- mdy("11/20/1993", tz = "America/New_York")
tz(x)

[1] "America/New_York"

PA 5.2: Jewel Heist

Just down the road in Montecito, CA several rare jewels went missing last fall. The jewels were stolen and replaced with fakes, but detectives have not been able to solve the case. They are now calling in a data scientist to help parse their clues.

Unfortunately, the date and time of the jewel heist is not known. You have been hired to crack the case. Use the clues below to discover the thief’s identity.

Submit the name of the thief to the Canvas Quiz.

Lab 5: Murder in SQL City

To do…

• Final Project Group Formation Survey – due Friday, 5/3 at 11:59pm.

• PA 5.2: Jewel Heist – due Saturday, 5/4 at 11:59pm.

• Lab 5: Murder in SQL City – due Saturday, 5/4 at 11:59pm.

• Read Chapter 6: Version Control

  • Check-ins 6.1 + 6.2 due Monday 5/6, at 10:00am